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Results of two studies testing the effectiveness of the Visual-Linguistic heading 
Program (Group A) as compared with that of a basal program using an overhead 
; proiector (Group B) and a basal program not using an overhead proiector (Group U 
are reported. Children from three cities in Minnesota. Florida, and California. l.oUU m 
the first study and 1.044 in the second, were pretested with the Metropolitan Reading 
Readiness Test and the Otis-Lennon Mental Ability Test and were tested al Mater 
times with Forms X and W of the Stanford Achievement tests. Results showed that 
, Group .A scored significantly higher than Group C on five of the 45 subtests (Word 
Reading. Paragraph Meaning, vocabulary. Spelling, and Word Study Skills). A slig 
numerical superiority involving significant differences among the 45 subtests favore 
v Group A over Group 8. In terms of total scores for each of the five subtests in the 
high-, middle-, and low-ability groups. Group A was significantly better than Group C in 

all five. Tables are included ror both studies. (MD) . 

% , 




ERIC 



i 



RE 0 0 1 823 EDO 305 36 



Y 




i : 

James I. Brown 
University of Minnesota !■ 
Department of Rhetoric l 
St. Paul, Minnesota 5510’. 



U. S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE 
OFFICE OF EDUCATION 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING IT. PO'NTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR POLICY. 



EVALUATING A VISUAL-LINGUISTIC, MULTI-MEDIA 



APPROACH TO PRIMARY READING. 



(Session on Evaluation of Materials for Reading ) 



It may come as a surprise to note that of the 27 first grade reading 
studies sponsored by the U. S. Office of Education in 1964-65, only one 
had the word visual in its title--"Evaluation of Levels-Designed Visual- 
Auditory and Related Writing Methods of Reading Instruction in First 
Grade." In this study, as the title indicates, the visual component is 
not isolated and explored separately. In fact the initial paragraph 
describing the Visual-Auditory method does not once use the word visual ♦ 
Obviously .even in the one study most concerned with the visual, that 
element is still net accorded predominant attention. Yet reading is a 
visual act-- the perceiving and comprehending of print. For that reason 
the visual dimension would seem to deserve particular attention and 
application if optimal help is to be provided for beginning readers. 
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But, important as any one element might seem, Dr. Russell G. Stauffer's 
comment in the October, 1966, issue of The Reading Teacher should be re- 
membered. In his editorial discussing the various methods for teaching 
reading in the U. S. Office of Education First Grade Studies he writes, 
"regardless of the criterion used there is no one method." 

With eclecticism as the guiding principle, the Visual-Linguistic 
Reading Program was built on a combination of methods and not on one 
only. Just as plant breeders have taken the best characteristics from 
several strains to develop superior varieties, so, in this program, five 
separate elements were fused to form a distinctly new approach. The 
program is intended to be more strongly oriented toward visual and 
linguistic elements than any other program yet devised for the teaching 
of reading. In addition, three other elements play an important role, 
making a total of five major components-- 1) the Visual, 2) the Linguistic, 

3) the Contextual, 4) the Listening, and 5) the Programmed. 

A program this different from the usual should, ideally, receive 
more intensive and extensive field and statistical testing than other 
programs. Fortunately the publishers were willing to provide just 
that--not one but two years of testing in three widely separated geo- 
graphic locations--public school systems in St. Paul, Minnesota; Tampa, 
Florida; and La Mesa, California. The present evaluation is based on 
data collected from those two test years, 1966-67 and 1967-68. 

The initial statistical testing during the 1966-67 school year was 
under the direction of the Supervisor of Reading and Special Learning 
Disabilities for the St. Paul Public Schools. The research design was 
structured to check the relative effectiveness of the Visual-Linguistic 



Program as well as to isolate and check the importance of the visual 
strand. Toward that end a three-fold grouping was used. Group A used 
the Visual-Linguistic Program, which incorporated frequent and carefully 
structured use of transparencies and the overhead projector. Group B 
used the regular basal reading program found in the school system but 
with an overhead projector to add a strong visual dimension. As a 
control, Group C also used the regular reading program but with no 
overhead projector available. 

In the fall of 1966 approximately 1,800 children from the three 
test centers were placed in one of the three experimental or control 
groups and given the Metropolitan Reading Readiness Test, followed in 
January, 1967, by Form X of the Stanford Achievement Test, and in May, 
1967, by Form W of the same test. The study ran for approximately 140 
days, following the pattern established by the United States Office of 
Education Studies on First Grade Reading Programs. 

Based on the data collected during the first test year, comparisons 
were made using the adjusted mean scores for each of the six subtests 
of the Stanford Achievement Tests in each of the three schools, 18 sub- 
test scores in all. By midyear, the Visual-Linguistic group (A) scored 
best in 6 of the 18 subtests, the overhead projector group (B) in 9 of 
the 18, and the regular program (C) in 3 of the 18. Differences at this 
point were, however, not significant. 

The end of the year testing did reveal statistically significant 
dif ferences--at the 5 per cent level or less (P<.05). The Visual- 
Linguistic group (A) out-scored to a significant degree the other two 
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groups in 5 out of 18 subtests, the overhead control group (B) scored 
significantly above the other two groups in 7 out of 18 subtests. The 
basal reading control group (C) scored significantly above the overhead 
projector group in only 1 out of 18 subtests and above the Visual - 
Linguistic group in no subtests. Other differences were not statistically 
significant. 

The comparisons between the basal reading groups, B and C, where 
the only difference was between using or not using the overhead pro- 
jector, provided statistically significant evidence of the importance 
of the visual element, one of the two matters of primary concern in this 
study. The findings indicate that use of the overhead projector brought 
measurably better results. The findings also provided pertinent evidence 
for justifying the strong emphasis on the visual factor in the Visual- 
Linguistic program. Despite the fact that the totally new Visual-Linguistic 
program demanded major adjustments on the part of the teachers, results 
were still significantly better than those in the regular basal programs, 
which involved no change from the usual teaching pattern. 

In addition to the statistical checking, use was made of teacher 
ratings--scales designed to reveal more clearly the strengths and weak- 
nesses in need of possible attention before the second year of testing. 

% 

A five-point rating scale was used, with two positive ratings- -Superior 
and Good, one neutral rating- -Average or the same as other programs, and 
two negative ratings- - Poor and Inferior, plus Not Answered ♦ 

The mid-year check showed 52.7 per cent of the teachers rated the 
program positively, 16. -8 per cent neutrally,^ 23.2 per cent negatively, 

7,2 per cant not answering. Of the 15 areas surveyed, the lowest rating 
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was given the teacher's manuals, no teacher raLing them superior to those 
used before and 44 per cent rating them inferior. Immediate steps were 
taken to re-work the teacher's manuals completely. 

The more complete ratings at the end of the year covered 22 areas 
and showed 61.2 per cent positive ratings, 24.4 neutral, 11.5 negative, 
and 2.7 not answering. While the end-of- the-year ratings still gave the 
teacher's manuals the lowest rating, only 18.1 per cent gave them the 
lowest rating- - Inferior, as compared with 44.4 per cent at midyear, a 
noticeable improvement when all the manuals had been used. 

» 

In answering the question, "How well does the Visual-Linguistic 
Reading Series seem to work with the superior student?" 90.9 per cent 
of the teachers gave it the top rating- - superior to other materials. 

The characters around which the series is built — Alphy, Canny Cat, 

Babby Big-Ear and Bob were, according to the ratings, characters the 
pupils related to extremely well, being rated superior by 31 to 77 per cent 
of the teachers. 

Teacher ratings and results from the statistical check provided exactly 
the guidance needed in making revisions to strengthen the program before . 
the second year of testing, which for 1967-68 was conducted by the Test 
Department of Harcourt, Brace and World, Inc. 

The same three schools were used, a total of 1044 pupils being tested-- 
three groups of 348 pupils, matched across groups in terms of their general 
mental ability and readiness of reading instruction. The Stanford Achievement 
Batteries, Form W, were used, the subtest on Arithmetic being omitted in the 



comparisons since this is out of the area of reading and differences between 
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treatment for that subtest were not significant. 

For the second year of statistical testing, high, middle and low 
ability groups instead of geographic differences were studied along 
with the same three- fold experimental and control variables. Each of 
the three groups of 348 pupils was divided into three cells of 116 
pupils each of high, middle, or low abilities. This meant 45 subtests, 
the 5 subtests dealing with reading for each ability grouping at each 
treatment . 

Pupils, in class units, were presumably assigned at random to 
one of the three treatment groups. Analysis of pre-test measures raised 
some doubt about the randomness of the assignment, thus necessitating 
some, adjustments in the data. It was intended that all pupils in the 

k ****** 

research program should take three tests : Metropolitan Readiness Test 

(MRT), 1966 Ed., Form A; Otis-Lennon Mental Ability Test (OLMAT), Primary 
II, Form J, 1967 Ed.; and Stanford Achievement Test, Primary I, Form W, 
1964 Ed., including the following subtests: (1) Word Reading, (2) Para- 

graph Meaning, (3) Vocabulary, (4) Spelling, (5) Word Study Skills, 

(6) Arithmetic. MRT and OLMAT were taken in September, 1967, and , K«.S' 
Stanford in May, 1968. 

Before the data were analyzed, two adjustments in number of pupils 
were made. First, all incomplete cases were eliminated. An incomplete 
case was defined as one for which one or more of the eight test scores 
(MRT, 0U1AT, and 6 Stanford scores) were missing; or for which there 
was not adequate information to determine whether the pupils belonged 
to Group A, B, or C for the entire academic year. Some switching of 
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pupils or teachers from one class to another during the year made such 
; determination impossible in some instances. 

t 

\ 

The second adjustment involved a pre-test matching operation. 
Preliminary analysis of the data made it apparent that pupils in Group 

j- A were noticeably superior to pupils in Groups B and C and pupils in 

! , # 

Group B were slightly superior to pupils in Group C in MRT and OLMAT 
scores. Therefore, pupils in the three groups were matched on the pre- 
test scores. It was decided that the most effective pre-test measure to 
use as a control would be a combination of MRT and OLMAT scores. Further, 

; it was decided that a simple sum of raw scores on MRT and OLMAT would 

provide about as effective a control score as any other combination. 

This simple sum yields a weighting of MRT to OLMAT of about 1.5 to 1. 

Such a weighting is in the direction suggested by the relative correlation 
i of MRT and OLMAT scores with the post- treatment Stanford test scores. 

Frequency distributions of MRT + OLMAT sum scores were prepared for each 
of Groups A, B, and C. The three groups were matched in five point 
intervals in the sum score. Then, since it was felt desirable to analyze 
the effect of the experimental treatments for pupils at differing initial 
ability levels, the three matched distributions were subdivided into three 

[ equal groups on the basis of MRT + OLMAT scores. 

f\ 

Table 1 summarizes the results of the matching, in terms of the 
pre-test .scores . Although pupils were matched specifically in terms 
of the MRT + OLMAT sum scores, Table 1 also presents summaries of those 
two sets of scores separately. 
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Table I 

Summary of Pre-test Scores for Sub-groups: 

Means and Standard Deviations for. MRT, OLMAT, and MRT + OLMAT 



TREATMENT GROUPS: 


A (348) 




13 (348) 


C (348) 


READINESS 

ABILITY 

LEVEL 


SCORES 


M SD 


M 


SD 


M SD 




MRT 


(116 cases) 
75.4 5.4 


(116 

74.7 


cases) 

5.6 


(116 cases) 
74.3 6.3 


HIGH 


OLMAT 


43.8 ' 2 


44.4 


4.0 


44.3 4.3 


(348) 


MRT + OLMAT 


119.2 7.3 


119.1 


7.4 


118.7 7.4 




MRT 


(116 cases) 
63.6 4.5 


(116 

62.2 


cases) 

5.5 


(116 cases) 
61.7 5.3 


MIDDLE 


OLMAT 


36.6 4.4 


38.2 


3.9 


38.4 4.3 


(348) 


MRT + OLMAT 


100.1 5.0 


100.4 


4.9 ' 


100.1 4.9 




MRT 


(116 cases) 
48.5 9.4 


(116 cases) 
47.1 8.6 


(116 cases) 
46.7 8.4 


LOW 


OLMAT 


28.9 6.1 


30.1 


6.3 


30.6 6.0 


(348) 


MRT + OLMAT 


77.4 11.3 


77.2 


11.4 


77.3 11.7 


TOTAL = 
1044 













Performance of the three treatment groups as a whole and by level of 
pre-test (readiness-ability) was evaluated by means of analysis of variance 
techniques. Data for each of the six Stanford subtests was subjected to 
two-way analysis of variance (fixed effects model with replications within 
cells). Comparison of mean scores by treatment across and within levels 
was accomplished by way of the Newman-Keuls procedure. Raw scores were 
used for all analyses of Stanford Tests. 
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Results 

The Visual-Linguistic Reading group (A) scored significantly higher 

than the control group (C) in five of the 45 subtests (Word Reading, 

V s - • . 

Paragraph Meaning, Spelling, and Word Study Skills). The control group 
(C), on the other hand, scored significantly higher than the experimental 
group (A) in not a single one of the 45 subtests. Furthermore there was a 
slight numerical superiority in the tallies involving significant dif- 
ferences among the 45 subtests, a superiority favoring the experimental 
group (A) over the control group (B). In terms of total scores for each 
of the five subtests in all three ability groups, the Visual-Linguistic 
group (A)was significantly better than the basal control group (C) in 
all five, as revealed in Table 2. 



L. 



Table 2 



Summary of Order of Treatment Means and 
Significance of Differences Between Treatment 
Means Across and Within Levels * 



Word Reading 


Best Worst 


High Ability 


ABC 


Middle Ability 


B A C 


Low Ability 


B C A 


Total 


B A C 


Vocabulary 


Best Worst 


High Ability 


B A C 


Middle Ability 


B A C 


Low Ability 


B A C 


Total 


B A C 



Word Study Skills 

High Ability 
Middle Ability 
Low Ability 

Total 



Paragraph Meaning 


Best.. 




Worst 


High Ability 


A 


B 


C 


Middle Ability I 


B 


A 


C 


Low Ability 


A 


B 


C 


Total 


A 


B 


C 


Spelling 


Best- 




Worst 


1 

High Ability 


A 


B 


C 


Middle Ability 


B 


A 


C 


Low Ability 


A 


B 


C 


Total 


A 


B 


C 



j.Best Worst 

A B C 

B A C 

B A C 

ABC 



(♦Treatment or group designations are arranged from highest to lowest 
mean scores, going from left to right--from best to worst, that is. 

Treatment designations not sharing an underline are significantly different. 

0*«.i, Av.M,} 

For example, in Wo**d Reading, ^ group A has the highest mean score, B the 
next highest and C the worst. A and B are not significantly different but 
A and B are significantly higher than C.) 
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The revisions and changes made alter the first: year of testing were, 
from all appearances, already being reflected in the statistical evidence 
from the second test year. As before, the addition of a stronger visual 
dimension through use of an overhead projector was enough to make the 
results for group B significantly better than those for group C. This 
was so, even though the three different school systems each used dif- 
ferent basal reading programs. Such findings add further weight to the 

l 

' . importance of the visual element in a reading' program. Despite the fact 

that the basal programs were for most of the teachers the program that 
they had most experience with, the new Visual-Linguistic approach got 
significantly better results even at the initial testing stage. 

Teacher ratings as well as statistical findings for the second year 
reflect the changes made after the first test year. 

\ As revealed in Table 3, the over-all ratings are noticeable higher. 

Table 3 

Summary of Comprehensive Teacher Ratings 



Better than 
Average 


Average 


Worse than 
Average 


Not 

Answered 


Initial ratings 
(1966-7) 


53% 


17% 


23% 


7% 


Final ratings 
(1966-7) 

(Year-end revisions) 


61% 


24% 


12% 


3% 


Final ratings 
(1967-8) 


70% 


18% 


9% 


3% 


For a more specific 


example, take 


the teacher 


' s manuals, 


the area 



receiving most attention in the end-of- the-year revisions. The teachers 
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were asked, "How would you evaluate the teacher's manuals for the Visual- 
Linguistic Series?" On the initial rating for the 1966-67 test year, 
no teacher rated the manuals superior, 11% rated them good, 7% rated them 
average, 30% poor, and 44% inferior, 8% not answering. By the end of that 
first year after using all the manuals, the ratings were somewhat better-- 
0% superior, 5% good, 23% average, 54% poor, and 18% inferior . The ex- 
tensive revisions led to greatly improved ratings for the second test 
year- -3 77. superior, 3 2% good, 217<> average, 57 0 poor, and 07 o inferior, 

5% not answering. 

Data from the first two years of testing is now being used as a 
basis for further changes and revisions to make up the final edition. 

The linguistic strand is being carefully re-developed to tap more fully 
the important contributions linguistic science has to make to reading 
instruction/, the visual strand being closely integrated with it for 
maximum effectiveness. 

Summary: 

1. On the five subtests from the Stanford Achievement Test related 
to reading there were significant differences between treatments on 
all five on the totals for high, middle, and low ability students, the 
Visual-Linguistic being significantly better than the basal reading 
control treatment and better but not significantly so from the over- 
head control. 

2. At the middle and low ability levels, differences tended to lack 
significance . 

3. Results for the Visual-Linguistic program tended to exceed those for 
the overhead projector control. 

4. The ove.rhead projector control (B) tended to exceed the Visual-Linguistic 
(A) with the middle and lowest ability groups, but not significantly. 
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